Repository navigation
Definition-first projects, the Map workspace, data tools, training and curation - #126
Merged
Merged
Conversation
Pumas v0.8 (feat/v080-consumer-api, 4c79cd8e) grows typed responses additively and makes error.outcome authoritative. Tuldok refused valid responses and misreported refusals. - Capabilities: validate only the fields, capabilities (chat, image generation, image-to-text) and option bounds Tuldok uses; ignore context_tokens, multimodal_embedding/decision, new formats, reasons, options and fields; select by name; drop list-length caps (1 MiB body bound instead). - Errors: classify by error.outcome, not HTTP status (404/409 are ordinary refusals); accept request_id null; keep unknown codes; show Pumas' bounded error.message in job/UI errors, capability refusals and pumas_error bodies on OpenAI-compatible routes (after redaction). - Results: ignore additive envelope/result fields. - Image-to-text: no fixed 2048 max_tokens cap; the live bound follows the served context. Image facts are checked only when advertised. - Model listing prefers GET /v1/served-models (profiles shown), falling back to /v1/models on 404. - Repin the six actual DTO schema exports, source-derived capability fixture and source.json to 4c79cd8e; update the consumer contract doc. The PR51 advertisement decoder stays closed because it is retained as evidence. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ement The gateway advertisement decoder was closed: any field Pumas added to the advertisement, or more than 64 entries in an open list, made the gateway disappear from scans. Required fields are now checked for presence and type only. Unknown fields are ignored and dropped from the decoded advertisement, so the stored build observation keeps only the known fields; list-length caps give way to the 64 KiB body bound. Missing required fields, wrong types, unsupported schema versions and malformed endpoints still fail, now as descriptor.Rejected with a reason naming the field. A gateway scan returns refused advertisements under `rejected` with their reasons, and the chooser shows them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The vision gateway fixture is projected from the Pumas DTOs, which now also advertise decision and multimodal_embedding. Pumas responses grow additively, so the browser flow asserts image_to_text is offered rather than an exact capability count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The approved redesign: a project folder holds only definitions, notes and result summaries; what is built from them stays in the database and a local product store. Includes the implementation spec for milestones 1 and 2 and the execution ledger for milestone 1. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Adds the tuldok_project package: strict reading of a project folder, JSON Schema and cross-file checks for the seven definition kinds, the uses-graph, and benchmark result summaries (statistics and results/*.csv files). Command line: python -m tuldok_project check FOLDER. No existing module is changed or imported except jsonschema. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A reference for people and agents who write a definition-first project folder: the layout, the seven definition kinds, result summary files, the rules that span files, and how to check a folder. Every example in it passes python -m tuldok_project check. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
store.open_project imports a checked folder into one schema per project (definitions as exact text plus jsonb, and the uses-graph as an edges table), attaches without re-checking when no definition file changed, and refuses a folder with errors without touching the database. Definitions can be set, removed and saved back to their files, with conflicts between unsaved edits and files changed on disk refused unless told to keep or discard. graph.states derives each definition's state from its own label and what it uses. The lab's 263 definitions open in 0.6 s into an empty database and in 0.01 s when nothing changed. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
python -m tuldok_project gains open, list, show, graph, status, set, remove, save and forget on top of the store. Each command opens the project (which imports any change on disk), does its work and releases the project's lock before it exits; reading commands take --json. The helpers the store shares with the reader and the checker are public names now. The contract describes the commands, and the ledger records milestone 2: the lab's 263 definitions open in under a second. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… milestone 3 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Results files are imported on open into a summary table, and are part of what makes a folder changed. A run's per-case values (tuldok_results_v1) can be imported; summaries for that subject are then computed from them, one overall row and one row per single group value, and save writes results/<benchmark>.csv back when it differs. Reading summaries, values and the overview is SQL only. On the lab project, 478,660 values import in 12 s and the recomputed summaries equal the files the lab's build script wrote, byte for byte. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
charts.chart_data answers a chart definition (ranking, tradeoff, progression, comparison) from the summary table with SQL only; a test makes every file read fail while it runs. render_svg draws the data as a self-contained SVG in the workspace design tokens, light or dark, and render_report builds one HTML page for a project: where it stands, every chart, the results overview and the definitions by kind. Command line: chart and report; results, values and chart accept a bare name or a reference. The contract documents results, values files, the chart forms and the commands. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
plan.build_plan lists, in dependency order, the steps that build one or more definitions: environments to set up, steps to build with the GPU, disk and time their definitions declare, and blocked targets with the definitions that cause the block and their reasons. Nothing is followed out of a blocked step, and a benchmark names subjects it would skip. Making a plan reads the database only; a test makes process creation and file opening fail while it runs. Command line: python -m tuldok_project plan FOLDER TARGET... [--all]. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
An environment definition can be fulfilled on this machine by connecting an existing interpreter or by installing one with uv from the definition's requirements. The check runs the interpreter once, reads the Python and package versions without importing them (torch is imported for a cuda environment to confirm the GPU is usable), and stores the result, so an environment is not set up, ready or failing. Plans show a ready environment as ready, not as work to do. Command line: environments, connect, check-environment, set-up (with --dry-run and --replace) and disconnect. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
snapshot.prepare copies the code a run needs from HEAD into a read-only directory under the project's local home, once per commit and set of code paths, and refuses a tree with uncommitted changes to tracked files unless a scratch run is asked for. tuldok.json gains code_paths, the parts of the repository a run needs. changed_since lists what changed after a commit, for telling when a product is out of date. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…gone launcher.run starts a command in its own session with its run id in the environment, watches wall time, disk use and a cancel file, stops it with SIGTERM then SIGKILL, and afterwards finds and kills anything still carrying the run id, reaping what it adopts as a subreaper. strays and stop_strays do the search alone for a run whose launcher is gone, and progress reads the log tails and the last tuldok_progress_v1 line. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
launcher.run now starts a small custodian (python -m tuldok_project.launcher) that is the child subreaper for one run: it starts the command, watches and stops it, and reaps everything it adopts before writing the outcome. The calling process no longer becomes a subreaper, so it is left with no zombie children and its own unrelated children keep their exit status. A run is stopped and swept when its caller dies, and a custodian that disappears without an outcome gives unknown after a sweep by run id. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
builds.execute runs a plan's steps in order from the commit's snapshot through the launcher: it resolves placeholders, uses the environment's checked interpreter, records each run and its product, checks declared outputs, and imports a benchmark's results. The project lock is released while a command runs, and a GPU step holds a machine-wide lock. A definition is now running, built, out of date or failed: out of date when its definition, its settings file, an input's product or its code changed since the run, or when something it uses is out of date. Plans show built steps as built and say why a step is out of date. Command line: build (with --scratch, --yes and --env), runs, products. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…imates Adds run (one run's details), progress (the last progress line and the log tails, with --follow), cancel (asks a running run to stop), and stop (kills what a run left behind when its builder is gone). A run can be named by an unambiguous prefix of its id. estimate runs each step's estimate command from the commit's snapshot under the launcher and stores the figures; plan shows them beside the declared ones while the definition is unchanged. It is the only way an estimate command is ever run. The contract gains a Building section covering snapshots, runs, products, states, limits, the commands and estimates. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
examples/tiny-project is the smallest real project for the new format: a dataset of noisy sine curves, a three-layer network trained on a GPU in under a minute, a benchmark against a predict-zero baseline, and two charts. Its README gives the commands to connect two environments, plan, build, follow progress and draw the charts. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
One project folder per Tuldok process, every operation a POST with its contract selected, builds asked for through the interface as requests the owner approves, and what was built differently recorded in section 12. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…quests operations.py declares the 36 operations a served folder offers: who may call each, its arguments and its result keys. service.py checks a call against that table, opens the project only for the moment it needs, and records every change asked for in a new activity table. A build asked for through the service is a build request. The owner's is queued at once; an agent's waits for the owner's approval, which holds to the commit and to the text of every definition in the plan. One worker per project runs queued requests in turn; builds.execute gains expect and stop so a changed definition or commit stops the build before the step starts. The command line gains requests, approve and decline, and its commands now wait a few seconds for a project that the app or a build has open. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
python app.py --project-folder PATH serves one definition-first project: POST /api/project/<name> for each operation, the project_folder capability profile, an owner client and the MCP bridge flag for it, and a page that shows the project's report. The catalog and old project capability manifests and the legacy discovery document are byte for byte unchanged; a test pins their digests. The agent client now passes on a project refusal's code and problems. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…d records Per type: kappas, Kendall's tau, interval IoU, matched IoU (boxes, polygons on a grid, rotated 3-D boxes, point sets, instance masks; Hungarian up to 64 objects a term, greedy above), semantic mask IoU, OKS, depth difference, track IoU with identity switches and fragments; a confidence filter and the disputed list. Tested against the test's own arithmetic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, and the decision kept annotation_sets, annotation_agreement, annotation_compare and keep_annotation (spec-t4 section 6), in the project_folder profile. A decision goes through set_annotation's own path; a kept sidecar is staged in the working copy and written to annotations/<task>/ by Save, which also removes those no saved line names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnotation type Two of a task's sets compared side by side or overlaid: masks, polygons, boxes, keypoints and depth drawn on the image, point objects and 3-D box wireframes on the point cloud, stretches under a series or a sound, tracks on an array's frames with a slider. The agreement, the most disputed records, Keep A / Keep B / Keep both / Reject both through keep_annotation, and a light correction of kept polygons and point objects. Parts are put in place only when they differ. New task offers the nine new types. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd the browser check of review examples/tiny-project gains make_scenes.py (about 4 KB of images, point clouds, frames and a series), a stand-in annotator, two tasks and four annotators. browser_project_workspace_annotation_kinds.cjs builds a task of eight types with two stand-ins each and checks the overlays, the agreement and disputed order, a decision kept and saved, both corrections, updates in place, light and dark at 1440 and 390. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t embedder Also: the tab bar no longer widens the page (the Compare tab made it 65 px too wide at 1000 px); on a phone the tabs take a scrolling row of their own; the annotate check expects the Compare tab. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…limits Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… served models Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… behind Tuldok's credentials Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tab and its drawer Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, agreement and review Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…'s credentials Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l after another A sweep definition (sweeps/<name>.json) names a recipe, the settings it varies, a strategy (grid, random, successive halving), one or two objectives from the trials' training logs and a budget. A build request for it alone covers the sweep, the recipe and what the recipe needs; the worker runs each trial as a derived recipe kept in the working copy, stops one that halving prunes or that outlasts the minutes, and records the runs and trials. Two reads, sweeps and sweep, and the map's groups of trials. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…parallel coordinates and on its frontier New sweep writes a sweep's text from a recipe's settings; the state block (run, trials by how they ended, budget used, running trial, best so far) is set in place on every reading; Approve and Decline when a request awaits approval; the trials table opens a trial in the drawer or on the Map, where trials carry the kind of change Sweep trial. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Also: the annotate check expects the Sweeps tab. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the lab's 206 recipes it asked definition() per recipe, which works out the whole project's states each time: 13 s. Now 0.09 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… full width In the 420 px drawer only two or three operations fit across. Open wide draws the same block, with the same state, over the page; Escape or Back to the drawer closes it. The structure check opens and closes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
On CI the environment has NumPy, so predict receives a float32 array and the answer encodes it again; the test assumed lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings Tuldok's November work to main (148 commits).
tuldok_log(autolog,export_graph), the model structure viewer, Compare and parallel coordinates with bootstrap intervals, generated annotation with agreement and review, curation by embeddings (EmbeddingGemma 2 default), serving a model on this machine, and hyperparameter sweeps.docs/plans/;docs/plans/training-and-curation/comparison.mdcompares this with the state before the last plan, anddocs/research/competitive-landscape/compares Tuldok with other tools.Checks on the final code: project suite 898 tests and agent interface 43 (4 skipped) pass; all 15 workspace browser checks pass.
Includes the work of #125 (Pumas 0.8 typed contract).
🤖 Generated with Claude Code